1. Field of the Invention
The present invention relates generally to computer graphics technology.
2. Background Art
Among the many functions that can be performed on personal and workstation computers, the rendering of images has become one of the most highly valued. The ever advancing demand for increasingly sophisticated image rendering capabilities has pulled the development of both hardware and software technologies towards meeting this end. Indeed, computer graphics applications have facilitated the introduction of multi-processors (or graphics units) into the designs of personal and workstation computers.
To increase rendering speed, computer graphics processes have been decomposed into standard functions performed in sequential stages of a graphics pipeline. At least one processor (or graphics unit) operates on each stage. As each stage completes its specific function, the results are passed along to the next stage in the pipeline. Meanwhile, the output of a prior stage (relating to the next frame in the sequence) is received. In this manner, the rendering speed of the overall process is increased to equal the processing speed of the slowest stage. Stages can be implemented using hardware, software, firmware, or a combination thereof.
Computer graphics pipelines can include, among other things, a geometry stage and a rasterizer stage. An application passes graphics data to a computer graphics pipeline. For example, an application may determine the image to be rendered and model the three-dimensional curvilinear form of each object in the image as a three-dimensional assembly of interconnected two-dimensional polygons (or any other type of primitive) that approximates the shape of the object. Each polygon is defined by a set of coordinates and an outwardly pointing vector normal to the plane of the polygon.
The geometry stage acts on the graphics data it receives from the application. The geometry stage often is further decomposed into more functional stages, each of which can have an associated processor to perform operations. For example, these stages can include, but are not limited to, a model and view transform stage, a light and shading stage, a projection stage, a clipping stage, a screen mapping stage, and others. The rasterizer stage uses the results of the geometry stage(s) to control the assignment of colors to pixels as the image is rendered.
As computer graphics has matured as a technology, approaches have been created to coordinate paths of development, to ensure compatibility among systems, and to reduce the amount of investment capital necessary to further the state of the art. These approaches allow designers a fair degree of leeway in choosing between hardware, software and firmware technologies to perform specific functions. Therefore, for a given hardware architecture, much of the current efforts in developing computer graphics centers on means to optimize the processing power of the given architecture.
The use of multiple processors in computer graphics hardware not only enables stages in a graphics pipeline to be processed simultaneously, but also allows for additional graphics pipelines for parallel processing. Graphics architectures have utilized these additional pipelines to process succeeding frames of images to support changes in a scene with time. Where the computer graphics hardware has “n” pipelines, each pipeline processes every nth frame in a sequence of frames. Each pipeline renders all of the objects and the background in a single frame. Often the outputs of the pipelines are multiplexed together to increase further the speed at which a sequence of frames is rendered. This process is known as temporal compositing, as image outputs from the pipelines are combined with respect to time.
While temporal compositing marks a major advancement in computer graphics performance, there are situations in which this approach may not optimize the processing power of the given architecture. Optimal temporal compositing, for a given number of graphics pipelines, depends on the relationship between the rendering speed of a given pipeline and the rate at which image outputs can be combined. A higher degree of image complexity, caused by additional features or geometrical entities (e.g., edges, polygons, etc) to improve the quality of the image, can reduce the speed at which a frame of an image is rendered by a graphics pipeline. This, in turn, lowers the rate at which image outputs can be combined.
Another problem posed by the temporal compositing process arises when the rendered images reside in an interactive environment. In an interactive environment, a user viewing a sequence of frames of images is permitted to supply a feedback signal to the system. This feedback signal can change the images that are rendered. In a temporal compositing system, there is a substantial delay between the time at which the user provides the feedback signal and the time at which the system responds to it. The user supplies the feedback signal at a particular frame to one of the pipelines in the system. Because the other pipelines are already in the process of rendering their pre-feedback frames, the system typically imposes a time delay to allow the other pipelines to complete their rendering of these frames before acting on the feedback signal.
Another type of compositing, called spatial compositing, also marks a major advancement in computer graphics performance. Like temporal compositing, spatial compositing relates to an approach to optimizing the utilization of multiple graphic units, and thus multiple pipelines. Rather than having each graphics unit or pipeline render an entire frame or a sequence of frames and having the output of each graphics unit combined temporally, spatial compositing uses each graphics unit to render a portion of each overall frame and combines the output of each graphics unit spatially with respect to the location of the rendered portion within the overall frame. By reducing the amount of graphics data (which may include geometry data) communicated to each graphics unit, spatial compositing increases the rate at which an overall frame is rendered.
In spatial compositing, optimization depends on the ability of the system to balance the processing load among the different graphics units. The processing load typically is a direct function of the size of a given subarea and a function of the rendering complexity for objects within this subarea. The more complex an object is, typically the more geometrical entities (and thus, geometric data) is used to represent the object. Load balancing strives to measure these variables and adjust, from frame to frame, the number, sizes, and positions of the subareas as necessary to balance the processing load (i.e., determine the current subarea boundaries). The cost of this approach is the necessity to communicate, in conjunction with each frame, the number, sizes, and positions of subareas being used for that given frame.
Where spatial compositing involves the use of a large number of subareas and/or complex graphics data, this can add substantially to the overhead information that must be communicated for each frame of spatially composited digital video. This situation compounds an already difficult problem with state of the art computer hardware technologies. As advancements in memory capacities and processor speeds have outstripped improvements in interconnect bus throughputs, data links have become bottlenecks of serious concern.